Multi protocol communication switch apparatus

ABSTRACT

A computing and communication chip architecture is provided wherein the interfaces of processor access to the memory chips are implemented as a high-speed packet switched serial interface as part of each chip. In one embodiment, the interface is accomplished through a gigabit Ethernet interface provided by protocol processor integrated as part of the chip. The protocol processor encapsulates the memory address and control information like Read, Write, number of successive bytes etc, as an Ethernet packet for communication among the processor and memory chips that are located on the same motherboard, or even on different circuit cards. In one embodiment, the communication over head of the Ethernet protocol is further reduced by using an enhanced Ethernet protocol with shortened data frames within a constrained neighborhood, and/or by utilizing a bit stream switch where direct connection paths can be established between elements that comprise the computing or communication architecture.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of the U.S. patent application Ser.No. 14/552,471 filed on Nov. 24, 2014 which is a divisional applicationof U.S. patent application Ser. No. 13/543,882 filed on Jul. 8, 2012which is a continuation of U.S. patent application Ser. No. 12/910,867filed on Oct. 25, 2010 which is a continuation of U.S. patentapplication Ser. No. 12/025,720 filed 4 Feb. 2008 which claims priorityof the provisional patent application Ser. No. 60/887,987 filed on Feb.2, 2007.

FIELD OF THE INVENTION

The present invention relates generally to the field of computing andcommunication architectures, and more specifically to an architecturefor processor and memory access using an integrated high speed packetswitched serial interface directly onto the same chip as the processorarrangement.

BACKGROUND OF THE INVENTION

The term computer architecture in a very broad sense connotes theinterconnection of a core set of functional units that include aprocessing subsystem that executes instructions and acts upon data, amemory subsystem that cooperates with the processing subsystem to enableselected data and instructions to be stored and transferred between thetwo subsystems, and an input/output (I/O) subsystem that allow at leastthe processing subsystem to exchange data and instructions with thenetwork and peripheral environment external to the computer. This coreset of functional units can be configured into different computer systemtopologies using various communication interconnection arrangements thatgovern the interchange of communications between the functional units.For example, a processor and its memories can be locally coupled in acircuit card or it could be geographically spread over a system chassisvia a back plane interconnection.

The Personal Computer (PC) represents the most successful and widelyused computer architecture. Architecturally, not much has changed sincethe PC was first introduced in the 1980s. At its core, a typical PC iscomprised of a single circuit board, referred to as a motherboard, thatincludes a microprocessor which acts as the central processing unit(CPU), a system memory and a local or system bus that provides theinterconnection between the CPU chip and the system memory chips locatedon the motherboard and the I/O ports that are typically defined byconnectors along an edge of the motherboard. One of the key reasons forthe success of the PC architecture was the industry-standardized mannerby which the components were interconnected.

A more recent example of a popular chassis-based computer architecturecan be found in the area of high performance computing (HPC). One of thearchitectural innovations in the HPC area has been the adoption ofserver blade configuration where one or more blades—such as serverblades, memory blades, I/O blades, PC blades are plugged into a commonrack that is based on industry standards. Instead of putting all of thechips for a computer system on a single motherboard, the functionalelements of the computer system are broken out into smaller circuitcards referred to as blades that are then coupled together by abackplane that routes the larges amounts of data among different blades.In most of these HPC blade configurations, the backplane fabric for thecommon rack has been implemented by a standardized parallel businterconnection technology such as the PCI bus. Breaking out thefunctional components onto blades permits more flexibility in terms ofconfigurations of components, while the use of a standardizedinterconnection such as the PCI bus permits blades from differentproviders to be configured together in the same common rack. Like thesuccessful PC architecture, the use of a standardized local or systembus interface such as the PCI bus has been critical to the success ofthe blade architecture for HPC and server computer systems.

One of the parameters that have a significant impact on the systemperformance and implementation is the memory access method used byprocessors. There are two fundamental architectures to access memory.One of the architectures is the Von Neumann architecture wherein oneshared memory is used to store instructions (program) and data with onedata bus and one address bus between processor and memory. Thisarchitecture requires instructions and data be fetched sequentiallyintroducing a limitation in operation bandwidth which is, often termedthe “Von Neuman Bottleneck”. The second architecture to access memory isreferred to as the Harvard architecture which uses physically separatememories and dedicated buses for their instructions and data.Instructions and operands can therefore be fetched simultaneously. Botharchitectures involve a bus or buses to transfer information between theprocessor and memory. It will be appreciated by those skilled in the artthat regardless of the processor and memory speeds, the speed ofinformation transfer between the processor and memory can substantiallyimpact the performance of the computer system.

While there have been significant strides with respect to the availableCPU power, memory capacity, and memory speeds for the individualcomponents of a computer system, progress in processor-memoryinterconnections and memory access in terms of the speed of the local orsystem parallel bus has lagged far behind. Processors and memories thatcan operate at upwards of 3 GHz clock are known, but local system busesthat can operate as a parallel bus interconnection at speeds that matchthe processor speeds are very rare as such high speed buses aredifficult to implement. For example, the system bus, referred to as thefront side bus, that is used to externally interface to a Pentium 4microprocessor chip operates slower than the speed of the processor.Conventionally, I/O devices external to the motherboard communicate overa slow speed I/O bus, such as the Peripheral Component Interconnect(PCI) Bus, that is connected to a chipset on the motherboard, referredto as a bridge, which in turn communicates with the CPU over the front,side bus. While this approach has worked well when I/O devicescommunicate at speeds that are much slower than the speeds of processorsand main memory, current developments in I/O technologies, such asInfiniband and Multi Gigabit Ethernet, can deliver I/O communications atrates approaching upwards of several gigabits per second. Thesedevelopments have blurred the conventional distinctions betweenCPU-memory and CPU-I/O transactions and negated the rationale forrelegating I/O communications to a separate, slower legacy I/O bus suchas the PCI bus.

One of the challenges in attempting to increase the speed of I/O buses,such as the PCI bus and PCI Extended (PCI X) bus, is that a parallel busarrangement is prone to problems of clock skew between data flowing inthe separate parallel data paths that may, for example, differ from eachotherby a very small path length. Clock recovery and data reconstructionprove to be increasingly problematic and unreliable as path lengths,data transfer speeds and/or the number of parallel paths are increased.Additionally, parallel buses take up considerable circuit board realestate.

Prior art solutions to the problems posed by increasing speeds onparallel buses for both front side buses and I/O buses have involved,for the most part, the use of proprietary protocols that are specific toa given provider of microprocessor chips and chipsets. For example, anadvanced version of the front side bus on the Athelon 64/FX/Opteron, byAdvanced Micro Devices, can operate at speeds approaching 1 Ghz for atheoretical bandwidth of 14400 MB/s for a parallel bus that is 32 bitswide. Unfortunately, this is a proprietary solution that is incompatiblewith the general trend of migrating to the adoption of industry widestandards that encourage vendors to develop products which areinteroperable with other vendors' solutions so as to reduce time andcost to market for new products.

The problem created by this divergence between processor speeds andmemory access speeds is well known and has been referred to in the priorart as the memory gap or memory wall problem. See, e.g., Cuppa et al.,“Organizational Design Trade-Offs at the DRAM, Memory Bus and MemoryController Level: Initial Results”, University of Maryland Systems &Computer Architecture Group Technical Report UMD-SCA-1999-2, November1999. The memory gap problem is further compounded by the need toaddress a large memory capacity. One solution employed in the prior artto overcome the memory wall/memory gap problem is to eliminate theparallel bus interface between the processor and memory and use a serialbackplane interface instead of a parallel bus like the PCI bus.

One early attempt to establish a standardized serial backplane interfacebetween processors and memories was the Scalable Coherent Interface.Gustavson, D. and Li, Q., “The Scalable Coherent Interface (SCI)”. IEEECommunications (August 1996). Unfortunately, this proposal was notwidely adopted.

More recently, proprietary high-speed serial interfaces betweenprocessors and memory have been developed by chip manufacturers, such asthe AMD® HyperTransport and the Intel® Fully buffered Dimm (FB DIMM).Other alternatives have been proposed in the form serial chip-to-chipinterfaces such as described by Trynosky, “Serial Backplane Interface toa Shared Memory,” Application Note: Virtex—II Pro FPGA Family, XILINX,Nov. 30, 2004 or and multiple single byte serial processor to memoryinterfaces as described by Davis, “The Memory Channel,” Summit ComputerSystems, Inc. Sep. 19, 2004.

The migration from parallel to serial interfaces among components in acomputing architecture is not unique to the processor/memory interface.Serial interfaces have also become the standard for almost all I/Ocommunication channels, including back planes. Advanced SwitchingInterconnect (ASI) switching fabrics that utilizes hierarchies andmultiple high speed clocked serial data lanes channels or proprietarypacket switched DMA techniques as described, for example, in U.S. Pat.No. 6,766,383. Industry standard I/O protocols, such as Infiniband,Fibre Channel and Gigabit Ethernet, can deliver I/O communications atrates approaching upwards of several gigabits per second.

While the speeds of a serial I/O protocol theoretically could approachthe speeds needed for the processor/memory interface, the communicationoverhead associated with serial I/O protocols has curtailed any seriousattempts to consider using serial I/O protocols as a basis for aprocessor/memory interface. Serial I/O communication protocols generallyhave larger packet and address sizes that are better suited foraccessing large amounts of data stored on disk or over a network. Thelarger packet and address sizes results in an increased communicationoverhead penalty. The processor/memory interface conventionally hasrequired the ability to transfer data between the processor and memoryfor a single address location, a requirement for which the overhead ofI/O transfers and protocols has been seen as massive overkill. Inaddition, there are many more transmission blocking and memorycontention concerns that need to be addressed for I/O communicationsthan for processor-to-memory interfaces.

Some alternatives that utilize a serial I/O interface protocol forbackplane connections instead of parallel bus interconnectiontechnologies have been proposed. U.S. Pub. Appl. No. 20050091304discloses a control system for a telecommunication portal that includesa modular chassis having an Ethernet backplane and a platform managementbus which houses at least one application module, at least onefunctional module, and a portal executive. In this patent application, a1000 BaseT (Gigabit Ethernet) backplane provides a packet-switchednetwork wherein each of the connected modules acts as an individual nodeon a network in contrast to a conventional parallel bus connection suchas a PCI bus.

U.S. Publ. Appl. No. 20060123021 discloses a hierarchical packagingarrangement for electronic equipment that utilizes an AdvancedTelecommunication Computing Architecture (TCA) arrangement of daughterboards in the for an Advanced Mezzanine Card (AMC) that areinterconnected with, a hierarchical packet-based interconnection fabricsuch as Ethernet. RapidIO, PCI Express or Infiniband. In thisarrangement, the AMCs in each local cube are connected in a hierarchicalconfiguration by a first, lower speed interface such a Gigabit Ethernetfor connections within the local cube and by a second, higher speedinterface such as 10 G Ethernet for connections among cubes.

The problems of Ethernet switched backplane architectures in terms oflatency, flow control, congestion management and quality of service arewell known and described, for example, by Lee, “Computation andCommunication Systems Need Advanced Switching,” Embedded IntelSolutions, Winter 2005. These issues have generally discouraged theadoption of serial I/O protocols for communications between processorsand memory even as such serial I/O protocols are being used in thesmaller physical dimensions of a circuit board or a computer orcommunication rack or cabinet having multiple cards/bladesinterconnected by a backplane. Instead, the trend has been to increasethe capacity of individual chips and the physical size of each of theserver blades in order to accommodate more processors and memory on asingle chip or circuit board, thereby reducing the need for processorand memory interconnection that must be mediated across the backplane.

As processor speeds, memory speeds and network speeds continue toincrease, and as the external I/O is increasingly capable of deliveringdata at rates exceeding gigabit speeds, the current architectures forarranging the subsystems within a computing and communicationarchitecture are no longer efficient. The problem of memory access likethe Von Newman and Harvard architectures, in the light of multipleprocessor cores with in a chip further aggravates the processor andmemory interconnect technology. There is therefore a need for acomputing and communication chip architecture that is not constrained bythe current architectural limitations and can provide a solution that iscompatible with industry configuration standards and is scalable tomatch the speed, capacity and processing core requirements of aconverged computing environment of the next generation computers andcommunications equipment.

SUMMARY OF THE INVENTION

The present invention is directed to a computing and communication chiparchitecture wherein the off-chip interfaces of processor and memorychips are implemented as a high-speed packet switched serial interfacesas part of each chip in a semiconductor package. In one embodiment, thehigh-speed packet switched serial interface is a gigabit Ethernetinterface implemented by a packet processor co-located with at least oneprocessor core within the chip package. The serial interface isconfigured to transfer data, address and control information, requiredto fetch and write data from and to an external memory device such as asystem main memory using a serial packetized protocol. Communicationsbetween at least one processor and the external memory device may bemediated by at least one bridge device capable of translating betweenmultiple serialized protocols and optionally a switch device adapted tomediate communications between on-chip entities such as processor cores,caches, and the packet processor, as well as the communications betweenon-chip entities and off-chip devices such as the system main memory.

In an exemplary embodiment, the packet processor is implemented as a onthe fly programmable bit stream protocol processor integrated as part ofthe chip. In one embodiment, a processor chip with cache can connect toa system or main memory chipset via a bit stream protocol processorincorporated as part of the microprocessor chip. In one embodiment theprocessor serial interface can be a 10 Giga bit Ethernet interface. Inthese embodiments, the protocol processor encapsulates the memoryaddress and control information like Read, Write, number of successivebytes etc, as an Ethernet packet for communication among theprocessor(s) and memory chips that are located on the same chip, or onthe motherboard, or alternatively on different circuit cards. In oneembodiment, the communication overhead of the Ethernet protocol isfurther reduced by using an enhanced Ethernet protocol with shorteneddata frames within a constrained neighborhood, and/or by utilizing a bitstream switch where direct connection paths can be established betweenelements that comprise the computing or communication architecture.

The above summary of the various embodiments of the invention is notintended to describe each illustrated embodiment or every implementationof the invention. The figures in the detailed description that followmore particularly exemplify these embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may be more completely understood in consideration of thefollowing detailed description of various embodiments of the inventionin connection with the accompanying drawings, in which:

FIGS. 1A, 1B, 1C, 1D, and 1E illustrate various configurations of frontside bus arrangements for prior art processor chipset architectures.

FIG. 2A depicts a chip architecture according to one aspect of thepresent invention wherein the processor chip package communicatesexternally via at least one serial line extending from a packetprocessor based parallel bus to serial interface converter located onthe die.

FIG. 2B is a block diagram representation of a multi-core processor chippackage according to one embodiment of the present invention that iscommunicatively coupled to the devices external to the chip via at leastone programmable serial interconnect extending from a switch and aparallel bus to serial interface module located within the chip package.

FIG. 2C is a block diagram representation of a multi-core processor chippackage according to one embodiment of the present invention that iscommunicatively coupled to the devices external to the chip via at leastone serial line extending from a module located within the package andadapted to function as a combination switch and a parallel bus to serialinterface.

FIG. 2D is a block diagram representation of a packet processor basedEthernet bridge that provides protocol translation and the serves as a“Southbridge” in a processor chip that features a unified computing,backplane, and network architecture.

FIG. 3A illustrates a more detailed block diagram of a packet processorbased parallel bus to serial interface converter that incorporates atoken based, point-to-point communication in Ethernet betweencommunications generating and consuming nodes in the system inaccordance with one embodiment of the present invention.

FIG. 3B illustrates a detailed block diagram of a packet processor basedparallel bus to a serial interface that converts to and from parallelbus communications and serial packetized communications based on apre-defined serial packet protocol in accordance with one embodiment ofthe present invention.

FIG. 3C is a schematic representation of a packet processor basedparallel bus to serial interface converter in which the serial packetprotocol output from the converter is programmable.

FIG. 4 illustrates an embodiment of the present invention incorporatedinto a three-dimensional chip architecture.

FIG. 5A is a block diagram of a processor chip package containing asingle processor “core” that communicated externally via at least oneserial line according to one embodiment of the present invention

FIG. 5B is a block diagram of a processor chip package containingmultiple processor “cores” each of which is placed in serialcommunication with a port on an external switch that in turncommunicates with devices external to the chip package.

FIG. 5C is a block diagram of a processor chip package containingmultiple processor “cores” each of which communicates with a multi-portparallel bus to serial interface converter contained within the chippackage and placed in serial communication with devices external to thechip package via at least one serial line.

While the invention is amenable to various modifications and alternativeforms, specifics thereof have been shown by way of example in thedrawings and will be described in detail. It should be understood,however, that the intention is not to limit the invention to theparticular embodiments described. On the contrary, the intention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIGS. 1A, 1B, 1C, 1D, and 1E illustrate various configurations of frontside bus (alternately “Channel”) arrangements for prior art processorchipset architectures. In each of these configurations, a clocked businterface 10 is used between the processor chip 15 and one or moresupport chips 20 for purposes of routing, data and instructions amongthe various elements of the computer architecture 5.

Conventional architectures feature a channel, variously referred to asthe Front Side Bus (“FSB”), the Processor Side Bus, the Memory Bus, theData Bus, or the System Bus, over which the CPU communicates with, forexample, a motherboard chipset such as the Northbridge and Southbridgecontrollers illustrated, for example, in FIG. 1D. The Northbridge 25interconnects the CPU 15 to the RAM memory 30 via the FSB. TheNorthbridge also connects peripherals such as the graphics card 35 viahigh speed channels such as the AGP and the PCI Express. The Southbridgecontroller 40 handles I/O including hard drives, USB, serial andparallel ports and external storage devices via other channels runningcommunication protocols such as Ethernet and PCI Express.

Currently, most front-side bus (FSB) speeds cannot deliver theperformance required of telecommunication and computing applicationsdesigned to comply with contemporary industry wide standards. Forexample, the PICMG® Advanced Mezzanine Card (AMC) specification definesthe base-level requirements for, a wide range of next generationhigh-speed mezzanine cards. For example, the AMC card interconnect isspecified at 12.5 Gbps per differential pair. Xilinx operates at 8 Gbpsand Fujitsu offers a 10-Gigabit Ethernet Switch. In comparison, theIntel Itanium 2 processor front-side bus (FSB) speed is approximately667 MHz, the AMD Opteron™ Front Side Bus frequency is approximately1.4-2.8 GHz, and the Intel Hub Architecture (IHA), which substitutes theMemory Controller and the I/O Controller for the Northbridge andSouthbridge controllers, features a system bus between the CPU and theMemory Controller that is capable of operating at speeds of 400 GHz,even though the dual RDRAM operate through the Memory Controller Hub(MCH) 25, to deliver a memory bandwidth of 3.2 GB/s as illustrated inFIG. 1D. FIG. 1E illustrates the IHA based multiprocessor architectureknown to the art.

One skilled in the art will appreciate that communication over the FSBand through the memory controller hub 45 of FIG. 1E, for example,introduces latency in RAM memory read operations. Furthermore, the RAMmemory access and I/O share the FSB bandwidth which can further degradeperformance of the FSB. Clearly, the telecommunication and highperformance computing applications designed to conform to theaforementioned industry specifications require an architecture that isfaster than the performance limits of the aforementioned interconnectsand is capable of operation under a wide range of industry standardprotocols such as Ethernet and PCI Express.

Referring to FIGS. 2A-2C, there is illustrated a multi-core processorarchitecture 50 according to a primary embodiment of the presentinvention. One aspect of the illustrated multi-core processorarchitecture 50 takes the form of a single physical package 55(alternately “Processor Chip Package”) that is received into a singleprocessor socket (not illustrated). This single physical package 55includes a plurality of execution cores (alternatively, computationalengines, or processing, engines) 60 but an external operating systemperceives the package as a single processor. In one embodiment, the core60 can be pin compatible with existing processor sockets. Each executioncore 60 includes its own processor-specific functional blocks such as,for example, caches, arithmetic logic units (ALU)s, priority interruptcontroller, architectural registers, pipeline prediction mechanisms, andinstruction set as seen in the illustrations of FIGS. 5A-5C. Eachexecution core is capable of independently executing programinstructions and a plurality of threads under the direction of theexternal operating system. In associated embodiments, the cores canexecute internal and/or external instructions in cooperation with theremaining core or cores in the package, an operating system candifferentiate between the services provided by each of the cores and thecores can access shared resources such as cache and external systemmemory 70 as seen in FIGS. 2A and 2C for example. In other embodiments,the operating system may be capable of supporting parallel executionamong multiple cores and each core, or various combinations of cores,can be seen by the operating system as separate parallel processingunits.

It will be appreciated that the present invention is not limited by anyparticular core or number of cores that might reside within a singlephysical package 55. In particular, the execution cores can be one ormore of the Smithfield core used in Intel's 90 nanometer Pentium D's andPentium Extreme Edition 840, the Presler core used in Intel's 65nanometer Pentium Extreme Edition 955 processor, AMD's 90 nanometerEgypt and Denmark cores. Other cores can be used within the scope of thepresent invention.

An important feature of the present invention is that data-communicationbetween the processor 55 and the system devices 80 occur via at leastone serial interconnect 90 mediated by a bridge-architecture 100 that inat least one embodiment communicates with a switch-architecture 105 asseen in FIG. 5C for example. The switch-architecture 5C is the gatewayvia which the rest of the devices 80 in the system and the processorcommunicate. In one embodiment, the bridge-architecture 100 andoptionally the switch-architecture 105 (alternately collectively“Parallel bus to serial interface converter”) are located on theprocessor die in an integrated configuration as illustrated in FIG. 2C.In such cases, one or more of the bridge architecture andswitch-architecture may be implemented in the form of additional core orcores on the die. Exemplary embodiments of the processor dieconfigurations are illustrated in FIGS. 5A-5C. In another configuration,the switch maybe located outside the die as illustrated in FIGS. 2A, 2B,5A and 5B. One of skill in the art will readily recognize that all suchconfigurations of the bridge-architecture and switch-architecture areincluded within the scope of the present invention. It must beemphasized that although the aforementioned embodiments are describedfor a multi-core architecture, the disclosed invention is equallyapplicable to the case where the processor package includes only onecore (single processor) and to the case where the bridge-architectureand the switch-architecture is a single module, such as the parallel busto serial interface converter 120 in FIGS. 2C and 5C for example.

In one embodiment, the bridge-architecture is implemented using a packetprocessor architecture as shown in FIG. 3A-3C. FIG. 3B is a specificembodiment of a typical packet processor according to the presentinvention. Communications from the processor transferred over a parallelbus 150, such as the data, address and control information related to a“write” command to external system memory issued by a processor core 60in an exemplary processor chip package 55, is processed by the packetprocessor portion 180 to generate serial packetized communications 155(165) that are transferred via one or more serial lines 90 outside thechip package 55. Serial communications 160 (170) received from outsidethe chip package 55 are processed by the packet processor portion 188into parallel communications transferred over parallel bus 150 to theprocessor as exemplified in FIGS. 3A and 3B. It will be appreciated thatthe serial-to-parallel transformations may be applied to communicationsbetween processor cores within the chip package, processor-core andexternal devices including other chip packages and I/O devices withinthe scope of the present invention. The functional blocks of FIGS. 3Aand 3B may be adapted according to a bitstream processor (BSP)architecture illustrated in FIG. 3C for instance. The Bitstreamprocessor is an on the fly programmable integrated packet processor,security engine and traffic manager using high performance pipelinedpacket switching architecture. The Bitstream processor may be physicallyimplemented as an additional “core”, integrated with other logic deviceson the processor die or on a stand-alone chip while remaining within thescope of the present invention.

In one embodiment of the present invention, the Bit Stream processorperforms a forward and reverse bridging function using a programmablepipelined architecture that provides high degree of flexibility foradaptation to legacy, existing and emerging board-level andnetwork-level data communication/signaling protocols. Each stage/blockwithin the pipeline has specific functions or responsibilities that makeavailable any relevant information to the subsequent blocks. As aconsequence, the architecture for each stage is different and isoptimized to handle a given function. Each stage can be dynamicallyprogrammed on a packet by packet basis while the processor corestransfer data/instructions by sending several bits at one time over aparallel communications link. The intra-core data/instructions usesignaling that is native to the processor-core and the associated systembus characterizing a vendor-specific CPU architecture such as forexample, signaling compatible with the front side bus by Intel, thehyper transport technology based interconnect protocol by AMD or otherproprietary/non-proprietary bus protocols. The Bit Stream processorbridges between the intra-processor protocol and one of a set ofboard-level or network level serial communication protocols. Upstreaminformation transfers to the processor from the bridge are parallelized,formatted and clocked so that they represent the native signaling usedby the processor cores. Responses from the cores (i. e. the downstreaminformation transfers) such as for example, the memory requests or othersystem requests, are serialized and packetized by the Bitstreamprocessor.

In one embodiment the Bitstream processor that processes the packetstakes the form as described in more detail in the previously identifiedco-pending application entitled “Omni-Protocol Engine for ReconfigurableBit-Stream Processing in High-Speed Networks.” The packet processing bythe Bitstream processor causes the packets to be bridged to a desiredboard level or network level protocol/bus-architectures and forwarded tothe switch-architecture. Exemplary protocols include, withoutlimitation, PCI-Express, 10 Gigabit Ethernet, Infiniband, AdvancedSwitching, RapidIO, SPI 4.2, XAUI and Serial I/O. Other protocols may beadvantageously used without limiting the scope of the present invention.

An alternate embodiment of the present invention, illustrated in FIGS.5A and 5B, contemplates an arrangement of the processor and bridgewherein the packet processor enables on-die connections for each of theplurality of protocols via separate ports comprised by one or moreprocessor pins. Each port is configured to provide serial input/outputto the processor in accordance with a specific pre-defined protocol.

In another related embodiment, the Bitstream processor is programmableto allow software based programming of the protocols characterizingcommunications at any particular serial interconnect or port. Each ofthe cores can be specialized to be application specific—such as packetprocessing for telecommunications, graphics engine functionality forgaming, and parallel computations for high performance computing. TheBitstream processor can be programmed to assign all traffic associatedwith a particular core to a specified port. In another embodiment of thepresent invention the aforementioned port can couple, to an AdvancedMezzanine Card (AMC) module and provide processor support to the modulewhere applicable or provide all or part of the Module ManagementController (MMC) functionality in an AdvancedTCA® (ATCA) based openmodular system architecture.

Referring again to FIG. 2C, there is shown a block diagramrepresentation of another feature of the present invention. As seen inFIG. 2C, the packet processor based bridge-architecture is coupled via aserial interconnect to a switch-architecture. The switch-architecture isa non-blocking switch that provides serial, high-speed, point-to-pointconnections in a cut-thorough mode between multiple devices and theprocessor. The switch-architecture may be implemented through merchantswitches such as, for example, the GigPCI-Express switch, model6468-8-port Gigabit Ethernet switch by DSS networks, or the MB8AA302020-port, 10 Gbps Ethernet (10 GbE) switch IC by Fujitsu MicroelectronicsAmerica.

In FIGS. 5B and 5C there is illustrated a multi core embodiment of theEther PC of the present invention with dual cores in which one of thecores is dedicated for communication applications. In this illustratedmulti core embodiment, there is separate program space and data space.The cores can access any space by switching between the two. The Data toI/O is switched. The switch allows a memory request originating at anexecution core to be switched to one or more external memory resources,thereby overcoming memory bandwidth limitations inherent in conventionalarchitectures where memory requests traverse a single data communicationbus to and fro from a single system memory resource.

Another embodiment of the present invention contemplates a switchingarchitecture implementation using the packet processor illustrated inFIGS. 2C and 5C. One of the features of such an embodiment is combinedbridge-switch architecture located on the processor die and capable ofproviding the services described above.

Another embodiment contemplates integrating the architectures disclosedin previously identified co-pending application entitled“Telecommunication and Computing Platforms with Serial Packet SwitchedIntegrated Memory Access Technology,” into a single die/processorpackage.

In one embodiment illustrated in FIG. 3A, the packet protocol processorallows line speed QoS packet switching which is utilized to implement asimple token based communication in Ethernet between the processor andthe devices in the system as set forth in previously identifiedco-pending application entitled “Enhanced Ethernet Protocol forShortened Data Frames Within a Constrained Neighborhood Based on UniqueID.” In this embodiment, the packetized communication over thebridge-switch-architecture is further specialized to speed-up sustained,point-to-point communications in the system. Each packet is providedwith a source address (SA) and destination address (DA) and E-type likeVLAN Tag for use in negotiating a unique token between end points on acommunication link. The E-type extensions may be, for example, Requestfor UNIQUE ID or TOKEN GRANT; data communication with the granted tokenand request to retire the TOKEN. Once the TOKEN has been granted, the SAand DA fields are used along with the E-type to pass short date. Thismay also be extended to include large blocks of data for STA, and SAS.In other embodiments, once a UNIQUE ID is negotiated between end-pointsand an intermediate node connecting these end-points, a fixed frame sizeis used to endow the link with predictable performance in transferringthe fixed frame and consequently meet various latency requirements. Forexample, the SA/DA pair could be used to transmit 12 bytes of data, 2E-Type bytes and 2 bytes TAG.

One of the embodiments to go along with multiple extended memories ismultiple caches. In one embodiment, the processor card is provided withtwo switchable caches (like two register files for threads). On a cachemiss, the processor switches over from the first cache to the secondcache to begin processing a second program thread associated with thesecond case. In another embodiment, there could be a cache per extendedmemory.

In one embodiment, control is provided as part of the extended Ethernetprotocol. This could also “add” to the CPU wait cycles if more than oneprocessor requests the same block of memory. In a sense that would be acomponent of latency because the processor and the instructionsscheduled for execution cannot distinguish between data localitydependent latency (speed of access and transfer) versus concurrencycontrol based data access “gap” because barring data mirroringconcurrent, access is not instantaneous access.

In another embodiment, the memory modules of the illustration of FIGS.2A and 2C comprise four Channel Fully-Buffered Dual Inline MemoryModules (FB-DIMM)s. FB-DIMM memory uses a bi-directional serial memorybus which passes through each memory module. The FB-DIMM transmitsmemory data in packets, precisely controlled by the AMB (Advanced MemoryBuffer) chips built into each FB-DIMM module. In one embodiment of thepresent invention, the four Channel FB-DIMMS are connected to 40 G linesand terminated to FB-DIMM lanes. The AMB is 10 lanes serial south boundand 14 lanes serial North bound. In terms of the AMC card of FIG. 2C,the AMB is configured to be a 16 Lane Fabric having less than 5 Gbpstotal bandwidth coming out of the memory Controller of FIG. 4A. Usingcommercial chips, such as for example, the Fujitsu Axel X (by FujitsuMicroelectronics America) which can provide speeds of 10 G per lane, theaforementioned requirements can be met by the use of a single 10 G lane.Additional bandwidth in excess of than 5 Gbps is provided by the use ofmultiple AMCs or multiple lanes. It will be appreciated that there isSerialization and De serialization on the DRAM end and serialization andde-serialization on the processor side. The latency penalty of theSwitch, and any overhead in the serialization and de-serializationmethods due to the serialization/de-serialization can be overcome in themanner set forth in the succeeding paragraphs.

In one embodiment, latency and contention/concurrency issues within theEthernet switched fabric are resolved within a “contained network.”Deterministic latency (tolerable margin jitter) through a “wellcontained network” (such as the packaging arrangement as describedherein) is indeed possible. Switching priority, dedicated ports (apseudo port to dedicated memory ports, communicating over Unique IDsbetween these ports and other techniques disclosed in the previouslyidentified co-pending application entitled “Enhanced Ethernet Protocolfor Shortened Data Frames Within a Constrained Neighborhood Based onUnique ID,” are advantageously utilized to overcome latency andcontention/concurrency related issues.

In another embodiment, the present invention can be adapted to support amesh architecture of processor-to-processor interconnection via theswitched Ethernet fabric. In one embodiment, N−1 connections are made toeach node with each node have 2 connections to all other nodes. In otherembodiments, different combinations of number of Ethernet ports/card,number of ports/switch and number of switches/packaging arrangement canprovide for various combinations of connections per node.

In another embodiment, the bit stream protocol processor enablesprioritized switching. In conjunction with the modular and scalablethree-dimensional chip architecture of the previous paragraph, thepresent invention allows the creation of an N-layered hierarchy ofmultiprocessors where N is both hardware independent and dynamicallyselectable by altering the prioritization afforded to different subsetsof processors in the bit stream, protocol processor mediated fabric.This embodiment enables the chip architecture to be configured as ashared memory model machine as well as a message passing modelmultiprocessor machine. Alternately, the architecture in accordance withone embodiment of the present invention may be configured as a server, astorage area network controller, a high performance network node in agrid computing based model, or a switch/router in a telecommunicationnetwork. It will be recognized that the same basic machine may beprogrammatically or manually altered into one or more of theaforementioned special purpose machines as and when desired.

Finally, while the present invention has been described with referenceto certain embodiments, those skilled in the art should appreciate thatthey can readily use the disclosed conception and specific embodimentsas a basis for designing or modifying other structures for carrying outthe same purposes of the present invention without departing from thespirit and scope of the invention as defined by the appended claims.

For purposes of interpreting the claims for the present invention, it isexpressly intended that the provisions of Section 112, sixth paragraphof 35 U.S.C. are not to be invoked unless the specific terms “means for”or “step for” are recited in the subject claim.

What is claimed is:
 1. An apparatus implementing packet switched serialprotocol communication chip architecture for integrated circuitry,comprising: a plurality of external serial communication protocol ports;and at least one programmable packet switch operably connected to thesaid plurality of external serial communication protocol ports; and, thesaid programmable packet switch is adapted to provide packet switchedserial protocol communication among the said plurality of externalserial communication protocol ports; and, the said plurality of externalserial communication protocol ports and the said programmable packetswitch are collocated on the semiconductor die package; wherein, atleast one external serial communication protocol port of the saidplurality of external serial communication protocol ports communicateswith external devices using a first packet switched serial communicationprotocol and, at least one external serial communication protocol portof the said plurality of external serial communication protocol portscommunicates with external devices using a second packet switched serialcommunication protocol and, at least one external serial communicationprotocol port of plurality of external serial communication protocolports communicates with external devices using a third packet switchedserial communication protocol and, the said programmable packet switchis further adapted to provide protocol translation between the saidfirst packet switched serial communication protocol, the said secondpacket switched serial communication protocol and the said third packetswitched serial communication protocol.
 2. An apparatus of claim 1wherein, the said programmable packet switch is further adapted toprovide point to point cut-through communication between any twoexternal serial communication protocol ports of the said plurality ofexternal serial communication protocol ports.
 3. The apparatus of claim2 wherein, the said first packet switched serial communication protocolor the said second packet switched serial communication protocol or thesaid third packet switched serial communication protocol is programmableby an external device, via one of the said plurality of external serialcommunication protocol ports.
 4. An apparatus of claim 3 wherein, thesaid first packet switched serial communication protocol or the saidsecond packet switched serial communication protocol or the said thirdpacket switched serial communication protocol is PCI express.
 5. Anapparatus of claim 3 wherein, the said first packet switched serialcommunication protocol or the said second packet switched serialcommunication protocol or the said third packet switched serialcommunication protocol is Ethernet.
 6. An apparatus of claim 3 wherein,the said first packet switched serial communication protocol or the saidsecond packet switched serial communication protocol or the said thirdpacket switched serial communication protocol is Serial Rapid IO.
 7. Anapparatus of claim 3 wherein, the said first packet switched serialcommunication protocol or the said second packet switched serialcommunication protocol or the said third packet switched serialcommunication protocol is Infiniband.
 8. An apparatus of claim 3 wherein, the said first packet switched serial communication protocol or thepacket switched serial said second packet switched serial communicationprotocol or the said third packet switched serial communication protocolis SPI 4.2.